INTERMODALITY  PERCEPTION 


by 


516  5 


WILLIAM  EDWARD  LEWIS 
B.  A.,  Wichita  State  University,  I967 


A  MASTER'S  THESIS 

submitted  in  partial  fulfillment  of  the 
requirements  for  the  degree 

MASTER  OF  SCIENCE 

Department  of  Psychology 

KANSAS  STATE  UNIVERSITY 
Manhattan,  Kansas 

1968 

Approved  by: 


Table  of  Contents 

Pattern  Presentation   

Method   

Subjects   

Apparatus   

Modalities   

Stimulus  presentation   

Task  

Experimental  Conditions   

Rate  of  presentation   

Start  point   

Patterns   

Pattern  presentation  methods  .... 

Experimental  Design   

Results   

Performance-Pattern  Identification    .  .  . 

Start  point  and  rate  of  presentation 

Pattern  presentation  methods  .... 
Pattern  Organization   

Preferred  start  point   

Non-preferred  start  point   

Practice.    Specific  or  General?   

Performance   

Organization   

Discussion   

Performance   

Organization   


This  study  was  designed  to  examine  the  effects  of  practice  on  the  percep- 
tion of  repeating  temporal  patterns. 

Stimulus  Patterns 

The  stimulus  patterns  used  were  repeating  sets  of  eight  dichotomous  ele- 
ments.   The  dichetomous  elements  were  stimuli  presented  either  to  S's  left  side 
(L)  or  to  S' s  right  side  (R) .    A  pattern  can  be  started  at  any  one  of  the  pat- 
tern elements,  but  since  the  patterns  are  continuously  repeated,  all  these  pat- 
terns are  logically  equivalent,  in  that  all  of  the  patterns  generate  the  same 
sequence  except  for  the  first  few  elements.    However,  they  are  not  perceptually 
equivalent.    Royer  and  Garner  (I966)  found  that  Ss  organize  and  describe  patterns 
by  starting  at  particixlar  elements  in  the  pattern  (preferred  start  points)  re- 
gardless of  the  actual  start  point  of  the  pattern.    Patterns  are  normally  organ- 
ized starting  at  a  series  of  identical  elements  (e.g.,  LLLL)  or  at  a  series  of 
alternating  elements  (e.g.,  LRLR) .    Moreover,  patterns  started  at  the  actual 
preferred  start  point  are  easier  to  identify  than  patterns  started  at  actual 
non-preferred  start  points.     (Garner  and  Gottwald,  I968) 

Pattern  Presentation 

Five  methods  of  presenting  the  patterns  were  used;  in  the  first  three 
methods,  the  pattern  could  be  identified  using  only  one  modality,  but  in  the 
remaining  two  methods,  information  in  two  modalities  must  be  combined  to  per- 
ceive the  pattern. 

Using  the  first  method  of  presentation  (Individual)  the  patterns  are  pre- 
sented in  a  single  modality,  either  the  auditory,  tactual  or  visual.    In  the 
second  method  (Compatible  Simultaneous),  a  pattern  is  presented  simultaneously 
in  two  modalities  so  that  if  a  left  element  appears  in  one  modality,  a  left 
element  also  appears  in  the  second  modality.    The  S  can  use  either  modality  or 
the  pair  or  modalities  to  identify  and  organize  the  pattern.    In  the  third 
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method  of  pattern  presentation  (Incompatible  Simultaneous),  a  pattern  is  pre- 
sented simultaneously,  but  reversed,  in  two  modalities  so  that  when  a  left 
element  is  presented  in  one  modality,  a  right  element  is  presented  in  the 
second  modality.    In  this  case,  the  patterns  are  informationally  compatible, 
although  the  pattern  elements  and  organizations  are  spatially  incompatible. 
This  method  of  presentation  is  a  case  of  stimulus- stimulus  incompatibility 
as  separate  from  stimulus-response  incompatibility  previously  studied  (Fitts 
&  Seeger,  1953) .     In  the  last  two  methods  of  presentation  the  pattern  is  given 
in  alternate  modalities  so  that  four  of  the  eight  elements  of  a  pattern  are 
presented  in  one  modality  and  the  remaining  four  elements  in  the  second  modality. 
For  example,  the  pattern  LLRRLRLR  could  be  represented  in  the  sequence-left 
light,  left  light,  right  light,  right  light,  left  tone,  right  tone,  left  tone, 
right  tone.    In  the  fourth  method.  Modality  Alternation  (4),  the  modalities 
alternate  every  four  elements,  and  in  the  fifth  method.  Modality  Alternation 
(2)  the  modalities  alternate  every  two  elements. 

Incompatible  simultaneous  presentation  and  modality  alternation  presen- 
tation represent  extremes  in  the  demands  on  using  information  from  two  modal- 
ities.   In  the  former  case,  the  emphasis  is  on  the  ability  of  two  senses  to 
operate  independently;  either  modality  or  the  pair  of  modalities  can  be  used 
to  identify  the  pattern,  but  only  one  modality  can  be  used  to  describe  the 
pattern.    In  the  latter  case,  the  emphasis  is  on  the  ability  to  shift  between 
modalities  and  then  to  synthesize  this  information;  both  modalities  must  be 
used  to  identify  the  patterns. 

Previous  work  (Handel  and  Buffardi,  in  preparation)  have  found,  using 
naive  Ss,  differences  among  methods  of  pattern  presentation.    The  aim  of  this 
study  is  to  investigate  the  effect  of  practice  on  pattern  identification  and 
organization  for  each  method  of  presentation. 


Method 

Sub.jects 

There  were  eight  Ss;  three  males  and  five  females.    Six  Ss  were  xmder- 
graduates  and  two  were  graduate  students.    Each  S  was  paid  $1.50  per  session. 
Apparatus 

Modalities. — The  two  stimuli  for  each  modality  were  perceptually  differ- 
ent; one  placed  on  S ' s  left  side  and  the  other  placed  on  S's  right  side.  The 
visual  stimxali  were  a  red  and  a  green  panel  light  mounted  1  foot  apart  on  a 
board  about  4  feet  in  front  of  S.    The  auditory  stimuli  were  a  1200  Hz.  and 
a  3000  Hz.  tone  presented  by  loudspeakers  6  feet  apart  and  4  feet  in  front 
of  S.    The  tactual  stimuli  were  two  Sherrick  vibrators  (1965),  one  held  in 
each  hand,  1  foot  apart.    One  vibrator  was  powered  by  a  12  V. ,  60  Hz.  source 
and  the  other  vibrator  was  powered  by  a  6  V.,  30  Hz.  source. 

Stimulus  presentation. — The  stimuli  were  controlled  by  a  tape  reader 
(Western  Union  IB)  which  activated  a  set  of  relays  to  switch  the  stimuli  on 
and  off.    Toggle  switches  controlled  the  choice  of  modalities.    Two  timers 
(Hunter,  Model  lllC)  were  used  to  time  the  sequence  and  trigger  the  tape 
reader.    A  counter  recorded  the  number  of  elements  presented. 

The  S  was  placed  in  a  soundproof  room.    The  E  observed  S  through  a  one- 
way mirror  and  communicated  with  him  by  intercom  system. 
Task 

The  S  observed  the  pattern  until  he  thought  he  knew  the  pattern.  Then, 
S  stopped  the  presentation  by  pressing  a  foot  switch  and  attempted  to  describe 
the  pattern  verbally  by  stating  the  left-right  sequence.    If  the  description 
was  incorrect,  the  pattern  continued  from  the  point  where  it  was  stopped.  If 
the  description  was  correct,  the  next  pattern  was  presented.    If  the  pattern 
was  not  correctly  described  after  560  pattern  elements  were  presented,  the 


next  pattern  was  presented. 
Experimental  Conditions 

Rate  of  presentation. — Two  rates  of  presentation  were  used;  one  or  three 
elements  per  second.    At  each  rate  of  presentation,  each  element  appeared  for 
the  first  one-third  of  the  inter-element  interval.    Thus,  at  the  rate  of  three 
elements  per  second,  each  element  lasted  111  msec,  and  the  interval  between 
elements  was  222  msec. 

Start  point. — Two  start  points,  determined  from  Royer  and  Garner  (I966), 
were  used  with  each  pattern.    The  preferred  start  point  was  at  an  element  which 
Ss  often  used  to  organize  the  pattern  and  which  also  led  to  fast  identification 
of  the  pattern.    The  non-preferred  start  point  was  at  an  element  rarely  used  by 
Ss  to  organize  the  pattern  and  which  led  to  slow  identification  of  the  pattern. 

Patterns. — Six  experimental  patterns  were  used  throughout  the  entire  exper- 
iment.   In  addition,  three  extra  patterns  were  used  only  during  the  first  five 
sessions  and  the  last  five  sessions.    The  patterns  at  both  start  points  are 
shown  in  Table  1. 

Pattern  presentation  methods. — Five  pattern  presentation  methods  were  used: 
individual  modality,  compatible  simultaneous,  incompatible  simultaneous,  modality 
alternation  (2),  and  modality  alternation  (4). 

If  a  pattern  was  presented  in  an  individual  modality,  the  pattern  was  pre- 
sented in  either  the  auditory,  tactual,  or  visual  modality.    If  a  pattern  was 
presented  in  a  pair  of  modalities  (the  remaining  four  methods  of  pattern  pre- 
sentation) ,  it  was  presented  in  either  the  auditory-tactual,  auditory-visual, 
or  tactual- visual  combination. 
Experimental  Desi^ 

The  entire  experiment  consisted  of  30  sessions.    Within  a  block  of  10 
sessions  all  the  experimental  conditions  were  presented  once  so  that  the 
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Table  1 

The  Patterns  uaed,  with  Preferred  and  Non-Preferred  Start  Points 


Patternc 


Experimental 


Extra 


Start 

Preferred 

LLRLLRLR 
LLLLRLRR 
LLLLRRLR 
LLRRLRLR 
LLLRRLLR 
LLRRLLRR 
LLLRLRLR 
LLLLRLLR 
LLLRLLRR 


Point 

Non-Preferred 
LRLLRLRL 
LLLRLRRL 
LLLRRLRL 
LRRLRLRL 
LLRRLLRL 
LRRLLRRL 
LLRLRLRL 
LLLRLLRL 
LLRLLRRL 
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entire  experiment  consisted  of  three  replications  of  the  experimental  con- 
ditions. 

In  one  session,  S  was  presented  36  patterns.    Patterns  were  grouped  by- 
twelves  and  each  group  was  presented  by  one  of  three  different  pattern  presen- 
tation methods.    The  six  patterns  were  presented  twice,  once  at  each  rate  of 
presentation  and  start  point.    In  the  first  five  sessions  and  in  the  last  five 
sessions,  Ss  were  presented  the  extra  patterns.    Each  extra  pattern  was  pre- 
sented two  times  by  each  of  the  three  pattern  presentation  methods  in  every 
session;  once  at  each  rate  of  presentation  and  start  point.    Therefore,  18 
extra  patterns  were  presented  and  a  total  of  54  patterns  were  presented  during 
these  sessions. 

The  order  of  presentation  of  all  experimental  conditions  was  counter- 
balanced across  Ss  for  each  block  of  10  sessions.    For  each  block  of  five 
sessions,  all  experimental  conditions  were  presented  to  4  Ss. 

Each  pattern  and  its  complement  (the  complement  of  LLRRLRLR  is  RRLLRLRL) 
were  presented  an  equal  number  of  times  and  when  using  alternation  presen- 
tation, one  modality  presented  each  segment  of  the  pattern  equally  often. 
Neither  of  these  controls  affected  pattern  identification  or  organization  so 
that  the  results  have  been  combined. 

Results 

Pattern  Identification 

The  measure  of  performance  was  the  number  of  elements  presented  until 
the  pattern  was  correctly  identified  (delay) .    The  median  delay  was  found  for 
each  pattern  at  each  experimental  condition.    An  analysis  of  variance  indicated 
that  the  only  significant  interactions  are  due  to  changes  in  the  rate  of  pre- 
sentation.   Increasing  the  rate  of  presentation  has  the  effect  of  making  hard 
conditions  extremely  difficult  and  increasing  the  difficulty  of  easy  conditions 
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only  slightly.    These  interactions  are  monotonic;  the  rank  order  of  difficulty 
of  the  conditions  does  not  change.    Therefore,  the  median  delay  of  each  pattern 
was  averaged  over  patterns  and  modalities.    Other  summary  statistics  (geometric 
means,  arithmetic  means,  standard  deviations)  produced  equivalent  results. 

Start  point  and  rate  of  presentation. — The  delays,  averaged  across  pat- 
terns and  methods  of  pattern  presentation,  for  each  start  point  x  rate  of  pre- 
sentation combination  as  a  function  of  practice  are  shown  in  Table  2. 

The  effect  of  practice  was  to  reduce  the  difference  in  performance. 
During  the  first  five  sessions,  the  mean  delay  at  the  easiest  condition  (pre- 
ferred start  point-1  element  per  second)  was  16  elements  and  the  delay  at  the 
hardest  condition  (non-preferred  start  point-3  elements  per  second)  was  88 
elements.    During  the  last  ten  sessions,  the  difference  between  these  condi- 
tions was  only  13  elements.    However,  the  rank  order  of  difficulty  of  the  Start 
Point  X  Rate  of  Presentation  conditions  remains  constant. 

Pattern  presentation  methods. — The  delays,  averaged  across  start  points, 
rates  of  presentation,  and  modalities  or  modality  pairs  of  each  method  of  pat- 
tern presentation  as  a  function  of  practice  are  shown  in  Table  3. 

For  every  block  of  five  or  ten  sessions,  there  were  reliable  differences 
between  pattern  presentation  methods.    During  the  first  five  sessions,  when  Ss 
were  naive,  a  series  of  t  tests  (d.f.=^)  indicated  that  incompatible  simul- 
taneous presentation  produced  the  best  performance.    There  was  no  difference 
in  performance  between  the  individual  modality,  compatible  simultaneous,  or 
successive  alternation  (4)  methods.    Successive  alternation  (2)  produced  the 
poorest  performance. 

By  the  second  block  of  five  sessions  and  for  the  remaining  sessions,  there 
were  no  reliable  differences  between  the  best  three  methods:  individual 
modality,  compatible  simultaneous  and  incompatible  simultaneous.  Successive 


6a 


Table  2 


The 

Avera-^e  Delay  until  Pattern  Identification  for  each  j 
Start  Point  Rate  of  Presentation  Combination  i 

j 

Experimental 
Sessions 

Rate  of 

Slov 

(One  Element  per  Sec.) 

i 

Presentation  ; 

Fast  ; 
(Three  Elements  per  Sec.)  1 

Preferred 
Start 
Point 

Non-Preferred  Preferred 
Start               '  Start 
Point  Point 

Non-Preferred  ; 
Start  j 
Point  j 

1-5 

16 

25 

88  1 

6-10 

11 

15 

2k 

1 

11-20 

10 

13 

18 

29  ! 

21-30 

9 

11 

15 

22 

Note:    The  delays  are  averaged  across  methods  of  pattern  presentation. 
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Table  3 

Average  Delay  until  Pattern  Identification  for 
each  Method  of  Pattern  Presentation 


Experimental 

Sessions  Method  of  Pattern  Presentation 


1-5 

Individual 
Modality 

3h 

Compatible 
Siinultajaeous 

32 

Incompatible 
Simultaneous 

2k 

Modality 
Alternation 

3k 

Modality 
Alternation 
(2) 

92 

6-10 

18 

15 

18 

28 

kk 

11-20 

15 

Ik 

lii- 

20 

32 

21-30 

13 

13 

13 

19 

29 

Note:    The  delays  are  averaged  across  start  points,  rates  of  presentation 
and  modalities. 
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alternation  (4)  produced  reliably  poorer  performance  than  the  three  easiest 
methods  and  the  successive  alternation  (2)  reliably  produced  the  poorest  per- 
formance. 

Pattern  Organization 

Analysis  indicated  that  pattern  organizations  were  similar  when  patterns 
were  presented  by  the  individual  and  simultaneous  methods  and  that  these 
differed  from  organizations  when  modality  alternation  methods  were  used.  In 
addition,  rate  of  presentation  did  not  affect  organization. 

Preferred  start  point . — If  a  pattern  began  at  a  preferred  start  point, 
the  pattern  was  usually  organized  beginning  at  that  element.    If  the  patterns 
were  presented  using  the  individual  or  the  simultaneous  methods,  88^  of  the 
organizations  were  at  the  preferred  start  point  during  the  first  five  sessions, 
and  increased  to  94^  by  the  last  five  sessions.    If  the  patterns  were  presented 
using  the  modality  alternation  methods,  90%  of  the  organizations  were  at  the 
preferred  start  point  during  the  first  five  sessions  and  increased  to  99/^  by 
the  last  five  sessions. 

Non-preferred  start  point. — For  the  patterns  started  at  the  non-preferred 
start  point,  the  percentage  of  organizations  at  the  non-preferred  and  preferred 
start  points  as  a  function  of  practice  are  shown  in  Table  4. 

When  patterns  were  presented  using  either  the  individual  or  simultaneous 
presentation  methods,  unpracticed  Ss  reorganize  the  pattern;  36^  of  the  organ- 
izations were  at  the  actual  start  point  (non-preferred)  and  57^  of  the  organ- 
izations were  at  the  preferred  start  point.    However,  with  practice,  patterns 
were  increasingly  organized  at  the  actual  start  point;  during  the  last  five 
sessions  79^  of  the  organizations  began  at  the  actual  start  point,  previously 
non-preferred,  and  1^%  of  the  organizations  began  at  the  preferred  start 
point. 


Table  k 


The  Percent  of  Organizations  at 
the  Non-Preferred  and  Preferred  Start  Points 
vhen  Patterns  began  at  Non-Preferred  Start  Points 
as  a  Function  of  Practice 


Experimental 

Sessions  Method  of  Pattern  Presentation 

Individual,  Modality  Alternation  (2) 

Compatible  Simultaneous  and 

and  Incompatible  Modality  Alternation  {h) 

Simultaneous  Presentation  Presentation 

Start  Point  Start  Point 

Non-Preferred       Preferred  Non-Preferred  Preferred 


1-5 

36 

57 

77 

13 

6-10 

51 

65 

2k 

11-15 

51 

kk 

68 

29 

16-20 

58 

35 

59 

3h 

21-25 

67 

28 

57 

3h 

26-30 

79 

19 

63 

32 

Note:    The  organizations  are  averaged  across  patterns,  modalities,  and 
rates  of  presentation. 


If  patterns  were  presented  using  either  modality  alternation  method,  the 
non-preferred  start  point  is  at  an  element  at  which  modalities  alternate  but 
the  preferred  start  point  is  at  an  element  at  which  the  modalities  do  not 
alternate.    Therefore,  organizations  at  the  non-preferred  start  point  reflect 
modality  organization,  not  pattern  organization,  and  organizations  at  the 
preferred  start  points  reflect  organizations  based  on  the  properties  of  the 
patterns. 

With  xmpracticed  Ss,  patterns  were  usually  organized  at  the  actual  non- 
preferred  start  point  and  rarely  organized  the  pattern  at  the  preferred 
start  point.    With  practice,  the  percentage  of  organizations  at  the  start 
point  decreased  to  about  60%  and  the  percentage  of  organizations  at  the  pre- 
ferred start  point  increased  mtil  about  33%  of  the  organizations  occurred 
at  this  element. 
Practice.    Specific  or  General? 

Performance.— The  generality  of  the  effect  of  practice  on  pattern  identi- 
fication was  assessed  by  presenting  three  extra  patterns  during  the  first  five 
sessions  and  again  during  the  last  five  sessions.    If  the  effect  of  practice 
was  general,  the  delay  of  these  patterns  should  be  equal  to  the  delay  of 
patterns  presented  throughout  the  experiment.    In  fact,  the  performance  of 
both  sets  of  patterns  was  nearly  identical.    During  the  first  five  sessions, 
the  average  delay  for  the  six  experimental  patterns  was  46  elements  and  the 
delay  for  the  extra  patterns  was  42  elements  (this  difference  was  not  reliable). 
During  the  last  five  sessions,  the  delay  for  both  sets  of  patterns  was  14 
elements. 

Organization.— The  organization  of  the  extra  patterns  can  also  assess 
the  effect  of  practice.    If  the  effect  of  practice  is  general,  then  the  organ- 
ization of  the  extra  patterns  should  be  similar  to  the  organization  of  the 
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experimental  patterns  during  the  last  five  sessions. 

If  the  patterns  begin  at  the  non-preferred  start  point  and  the  patterns 
are  presented  by  the  individual  or  simultaneous  methods,  35^  of  the  organ- 
izations were  at  the  non-preferred  start  points  during  the  first  five  sessions 
and  B5%  were  at  the  non-preferred  start  points  during  the  last  five  sessions. 
If  the  patterns  were  presented  by  the  modality  alternation  methods,  59^  of  the 
organizations  were  at  the  non-preferred  start  points  during  the  last  five 
sessions.    Thus,  organization  of  the  extra  patterns  is  practically  identical 
to  the  organization  of  the  experimental  patterns  during  the  last  five  sessions. 

Discussion 

Performance 

With  unpracticed  Ss,  the  incompatible  simultaneous  presentation  produces 
the  best  performance;  S-S  incompatibility,  unlike  S-R  incompatibility,  (Fitts 
&  Seeger,  1953)  aids  performance.    One  possible  way  in  which  Ss  can  use  the 
incompatible  S-S  information  is  to  learn  the  alternation  sequence.    For  example, 
the  pattern  LLRRLRLR  can  be  identified  as  2,2,1,1,1,1  and  then  either  "left"  or 
"right"  can  be  substituted  in  the  pattern  organization.    In  addition,  with 
unpracticed  Ss,  modality  alternation  (4)  is  no  harder  than  individual  modality 
or  compatible  simultaneous  presentation.    Methods  of  pattern  presentation  in 
which  information  in  two  modalities  must  be  integrated  does  not  necessarily 
produce  poorer  performance.    However,  with  practice,  presentation  methods  in 
which  a  pattern  can  be  identified  using  one  m.odality  are  easier  than  methods 
in  which  a  pattern  information  must  be  integrated. 
Organization 

The  effect  of  practice  is  to  make  perceptual  organization  more  flexible. 
If  patterns  begin  at  non-preferred  start  points  and  are  presented  using  the 
individual  or  simultaneous  methods,  practiced  Ss  organize  the  pattern  at  the 
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actual  starting  point.    The  Ss  organize  at  the  actual  starting  point  patterns 
which  have  been  presented  often  (experimental  patterns)  and  patterns  which 
have  been  presented  only  once  previously  (extra  patterns) .    The  Ss  have  not 
simply  memorized  the  patterns  starting  at  different  elements. 

If  a  pattern  begins  at  a  non-preferred  start  point,  and  is  presented 
using  the  modality  alternation  methods,  unpracticed  Ss  organize  by  the  modality 
alternations,  but  practiced  Ss  begin  to  organize  by  the  structural  properties 
of  the  pattern.    The  pattern  had  been  previously  hidden  by  the  modality  alter- 
nations. 

For  all  methods  of  pattern  presentation,  practice  leads  to  different  types 
of  pattern  organization;  either  from  pattern  organization  to  element  organi- 
zation or  from  modality  organization  to  pattern  organization. 

These  resiilts  provide  a  rationale  for  the  sensory  channel  versus  type  of 
stimuli  organization  investigated  by  Broadbent  (1958),  Broadbent  and  Gregory 
(1964),  and  Interna  and  Trask  (I963) .    Broadbent  (1958)  demonstrated  that  two 
different  digits  presented  simultaneously  to  the  left  and  right  ears  are 
organized  by  ear,  and  Yntema  and  Trask  demonstrated  that  a  digit  and  a  word 
simultaneously  presented  to  the  left  and  right  ears  are  organized  by  type  of 
item.    The  Ss  in  this  experiment  show  both  types  of  organization  when  patterns 
are  presented  using  the  modality  alternation  methods.    Unpracticed  Ss  organize 
by  modality  (by  ears)  and  practiced  Ss  organize  by  pattern  (by  type  of  item) . 
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Abstract 


The  effect  of  practice  on  the  perception  of  temporal  patterns 
presented  in  the  visual,  auditory,  tactual  modalities  and  combinations 
of  these  modalities  was  investigated.  With  practice,  differences  in 
performance  between  experimental  conditions  decrease,  but  the  rank 
order  of  difficulty  of  the  conditions  was  generally  consistent.  In 
addition,  after  practice,  pattern  organization  was  more  flexible;  the 
structural  properties  of  the  pattern  no  longer  dominate  organization 
and  Ss  can  organize  by  starting  at  any  pattern  element .  The  effect  of 
practice  was  quite  general;  patterns  used  only  once  previously  can  be 
Identified  as  easily  as  well  practiced  patterns  • 


